The present invention relates to a method and apparatus for evaluating processors for architectural compliance and, more particularly, to a method and apparatus which imposes a predetermined amount of constraint, or structure, on a randomly generated case being used to evaluate or test a processor for architectural compliance.
In the industry of microprocessor design and implementation, microprocessors are tested to determine whether or not they are architecturally compliant, i.e., whether or not they have been designed correctly to meet all architectural requirements. Presently, in order to test the microprocessor, exceptional events are generated, such as traps, faults and aborts. These types of events place the microprocessor in an interesting state and often are difficult for the microprocessor to handle properly. Therefore, it is desirable to test the microprocessor to ensure that it is capable of properly handling these types of events.
When an exceptional event occurs, the microprocessor transfers control to a trap handler, which is code that resides at an architecturally-defined address. FIG. 1 is a block diagram that illustrates exception handling during normal execution of an application program. FIG. 1 depicts a program being executed by hardware 2 running operating system 3. When an event 6 occurs during execution of the instructions comprised by the program, the normal flow of control of execution is transferred, as indicated by arrow 8, from the point of interruption 5 to the event handler routine 7. The input to the event handler routine 7 corresponds to the instruction pointer at the point of interruption 5 and the hardware register state at the point of interruption.
Once the event has been handled by the event handler routine 7, execution resumes from the point of interruption 5 in accordance with the normal flow of control of execution. The first few instructions of a trap handler are executed with the processor in a very interesting state. A variety of state machines are in the process of responding to the event.
In the past, in order to test architectural compliance of the microprocessor, random code generators have been utilized to generate specific code and to simulate execution of the code by the microprocessor. The specific code may or may not cause an event to occur. When an exceptional event does occur during the simulation, the various states of the registers of the microprocessor during the simulations are known. The correct states of the registers, i.e., what the values in the registers should be, is also known since synchronous events are predictable. The values obtained from the simulation may then be compared against the correct values to determine whether or not the microprocessor handled the event properly.
By utilizing a random code generator to randomly generate instructions to be executed by the microprocessor, the amount of randomness interjected into the testing process is increased which, in turn, increases the robustness of the testing process. However, the amount of randomness inherent in the process is constrained. When an instruction results in the occurrence of a particular exceptional event, the trap handler instructions associated with the particular event are executed in the same manner each time the trap handler is called. For example, if the trap handler is to perform a series of tasks which cause five different registers to be set to certain values, these tasks are performed in the same order and in the same manner each time the trap handler is called. Therefore, there is no randomness with respect to the manner in which the trap handler itself performs its functions.
It would be desirable to randomize the manner in which the trap handler performs its tasks to thereby increase the overall randomness of the testing process. Accordingly, a need exists for a method and apparatus for randomizing the performance of the tasks by the trap handlers. The present invention allows the manner in which the trap handler tasks are performed to be further randomized, as described below in detail. This ability to further randomize the trap handler tasks allows the processors to be tested more rigorously for architectural compliance due to the fact that the processor is not responding to an event in the same, identical manner each time the event occurs.
It would also be desirable to generate other types of xe2x80x9cnon-exceptionalxe2x80x9d events in order to evaluate the ability of a processor to properly respond to these types of events. For example, it would be desirable to generate and simulate a call-return instruction pair in order to determine whether a processor properly handles this type of event. Processors currently exist that implement mechanisms for performing branch prediction. These types of events, i.e., branches, could be generated and simulated in order to test the ability of the processor to properly respond to them. Since the execution of the call instruction must always be performed before the execution of the return instruction, the manner in which this type of event is generated is structurally constrained. It would be desirable to allow these types of events to be generated with some degree of randomness so that the processor is not performing the tasks associated with an event in the same, identical manner each time the event occurs. However, the degree of randomness must be constrained a sufficient amount to ensure that the event is generated with the necessary amount of structure.
Accordingly, a need exists for a method and apparatus for generating and simulating various types of cases requiring some structure while adding a degree of randomness to the case without destroying the structure of the case.
The present invention provides a method and apparatus for testing architectural compliance of processors wherein various types of cases requiring some structure can be simulated and a degree of randomness can be added to the case without destroying the structure of the case. The apparatus comprises a computer capable of being configured to execute a testing program. When the computer is executing the testing program, the computer generates instructions and simulates execution of the instructions in a processor. During simulation, the computer detects when simulation of an instruction has caused an event to occur in the processor. The computer identifies the event that has occurred and generates a list of atoms and stores the list in a memory device in communication with the computer. Each of the atoms in the list corresponds to a description of a particular event handler task to be performed by the processor in response to the occurrence of the event. The atoms are then read out of the list and instructions corresponding to the atoms are generated and simulated to handle the event. The sequence in which these instructions are simulated can be randomized by either inserting the atoms into the list with a degree of randomness or by reading the atoms out of the list with a degree of randomness. The degree of randomness is added to the case without destroying the structure of the case.
The structure of the case is protected by configuring the list with unordered and ordered atoms. The list comprises at least one unordered atom and at least one ordered atom. The unordered atoms correspond to one or more event handler tasks that can be performed in any order. The ordered atoms correspond to one or more event handler tasks that must be performed after the tasks associated with the unordered atoms have been performed. For example, the ordered atom may correspond to an instruction which causes a return to the point of execution at which the event handler was called. The computer reads the atoms out of the list and generates and simulates one or more instructions corresponding to each of the atoms. The instructions corresponding to the unordered atoms are simulated before simulation of the instructions corresponding to the ordered atoms.
In accordance with the preferred embodiment of the present invention, the computer randomly selects locations in the list in which the unordered atoms are to be stored and stores the unordered atoms at the selected locations. The computer stores the ordered atom at a particular location in the list, preferably at the end of the list. The computer reads the atoms out of the list in such a sequence that the ordered atom is the last atom read out of the list. The computer generates and simulates instructions corresponding to each of the atoms as the atoms are read out of the list. Preferably, the computer reads the atoms out of the list in sequence from top to bottom so that the ordered atom is the last atom read out of the list.
Preferably, the test program is a random code generation program. In accordance with the preferred embodiment, after an unordered atom has been read out of the list and before an ordered atom has been read out of the list, the random code generation program generates at least one random instruction and simulates the random instruction. This allows the sequence in which the event handler tasks are simulated to be further randomized. In accordance with this embodiment, after an event has been detected, the random code generation program makes a determination as to whether a random instruction should be generated and simulated or whether an atom should be read out of the list and an instruction corresponding to the atom should be generated and simulated. On some occasions, the random code generation program determines that a random instruction should be generated and simulated and on other occasions the random code generation program determines that an atom should be read from the list and an instruction corresponding to the atom should be generated and simulated.